Examination apparatus, examination method, recording medium storing an examination program, learning apparatus, learning method, and recording medium storing a learning program

ABSTRACT

Provided is an examination apparatus including a target image acquiring section that acquires a target image obtained by capturing an examination target; a target image masking section that masks a portion of the target image; a masked region predicting section that predicts an image of a masked region that is masked in the target image; a reproduced image generating section that generates a reproduced image using a plurality of predicted images predicted respectively for the plurality of masked regions; and a difference detecting section that detects a difference between the target image and the reproduced image.

BACKGROUND 1. Technical Field

The present invention relates to an examination apparatus, anexamination method, a recording medium storing thereon an examinationprogram, a learning apparatus, a learning method, and a recording mediumstoring thereon a learning program.

2. Related Art

Conventionally, an appearance examination of an examination target isperformed to judge acceptability of this examination target. Forexample, in an appearance examination of a probe pin for testing asemiconductor, an image of the probe pin is captured and digitized, andthe digitized data is evaluated using predetermined rules to judgeacceptability of the probe pin.

However, when examining the examination target, there is a desire tomake it easy to understand the state of the examination target.

SUMMARY

In order to solve the problem above, according to a first aspect of thepresent invention, provided is an examination apparatus. The examinationapparatus may comprise a target image acquiring section that acquires atarget image obtained by capturing an examination target. Theexamination apparatus may comprise a target image masking section thatmasks a portion of the target image. The examination apparatus maycomprise a masked region predicting section that predicts an image of amasked region that is masked in the target image. The examinationapparatus may comprise a reproduced image generating section thatgenerates a reproduced image using a plurality of predicted imagespredicted respectively for a plurality of masked regions including themasked region. The examination apparatus may comprise a differencedetecting section that detects a difference between the target image andthe reproduced image.

The difference detecting section may compare the target image to thereproduced image in every predetermined region, to calculate a degree ofthe difference in every predetermined region.

The examination apparatus may further comprise a judging section thatjudges the examination target to be unacceptable if the degree ofdifference does not satisfy a predetermined quality standard.

The judging section may judge the examination target to be unacceptableif a largest degree of difference, among the degrees of difference ofevery predetermined region, exceeds a predetermined threshold value.

If the judging section judges the examination target to be unacceptable,the judging section may predict an electrical characteristic of theexamination target from the target image obtained by capturing theexamination target that was judged to be unacceptable, and confirms thatthe examination target is unacceptable if the electrical characteristicdoes not satisfy a predetermined quality standard.

If the judging section judges the examination target to be unacceptable,the judging section may predict an electrical characteristic of theexamination target from the target image obtained by capturing theexamination target that was judged to be unacceptable, and determinesthat the examination target is acceptable if the electricalcharacteristic satisfies a predetermined quality standard.

The difference detecting section may output a detection map in which adisplay attribute differs in every predetermined region, according tothe degree of difference.

The difference detecting section may calculate the degree of differencebased on a Euclidian distance between the target image and thereproduced image.

The target image masking section may sequentially mask one cell at atime among a plurality of cells obtained by dividing the target image,and the reproduced image generating section may generate the reproducedimage using a plurality of predicted images predicted respectively fordifferent cells.

The target image acquiring section may acquire an image obtained byperforming a grayscale conversion on the captured image of theexamination target, as the target image.

The target image acquiring section may acquire an image obtained byperforming object detection on the examination target in the capturedimage of the examination target to narrow a target region, as the targetimage.

According to a second aspect of the present invention, provided is anexamination method. The examination method may comprise acquiring atarget image obtained by capturing an examination target; masking aportion of the target image; predicting an image of a masked region thatis masked in the target image; generating a reproduced image using aplurality of predicted images predicted respectively for a plurality ofmasked regions including the masked region; and detecting a differencebetween the target image and the reproduced image.

According to a third aspect of the present invention, provided is arecording medium storing thereon an examination program. The program,when executed by a computer, causes the computer to function as a targetimage acquiring section that acquires a target image obtained bycapturing an examination target; a target image masking section thatmasks a portion of the target image; a masked region predicting sectionthat predicts an image of a masked region that is masked in the targetimage; a reproduced image generating section that generates a reproducedimage using a plurality of predicted images predicted respectively for aplurality of masked regions including the masked region; and adifference detecting section that detects a difference between thetarget image and the reproduced image.

According to a fourth aspect of the present invention, provided is alearning apparatus. The learning apparatus may comprise a training imageacquiring section that acquires a training image; a training imagemasking section that masks a portion of the training image; a predictivemodel that receives the masked training image and output a model imageobtained by predicting the training image; and a model updating sectionthat updates the predictive model based on an error between the trainingimage and the model image.

According to a fifth aspect of the present invention, provided is alearning method. The learning method may comprise acquiring a trainingimage; masking a portion of the training image; inputting the maskedtraining image to a predictive model and outputting a model imageobtained by predicting the training image; and updating the predictivemodel based on an error between the training image and the model image.

According to a sixth aspect of the present invention, provided is arecording medium storing thereon a learning program. The learningprogram, when executed by a computer, causes the computer to function asa training image acquiring section that acquires a training image; atraining image masking section that masks a portion of the trainingimage; a predictive model that receives the masked training image andoutput a model image obtained by predicting the training image; and amodel updating section that updates the predictive model based on anerror between the training image and the model image.

The summary clause does not necessarily describe all necessary featuresof the embodiments of the present invention. The present invention mayalso be a sub-combination of the features described above.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 shows a block diagram of the examination apparatus 100 accordingto the present embodiment.

FIG. 2 shows an example of a flow for examining an examination targetwith the examination apparatus 100 according to the present embodiment.

FIG. 3 shows an example of a target image 310, masked images 320,predicted images 330, and a reproduced image 340 in the examinationusing the examination apparatus 100 according to the present embodiment.

FIG. 4 shows an example of the examination result in a case where theexamination target is acceptable, in the present embodiment.

FIG. 5 shows an example of examination results in a case where theexamination target is unacceptable.

FIG. 6 shows an example of a block diagram of a learning apparatus 600according to the present embodiment.

FIG. 7 shows an example of a flow of the learning of the predictivemodel 630 by the learning apparatus 600 according to the presentembodiment.

FIG. 8 shows an example of a computer 2200 in which aspects of thepresent invention may be wholly or partly embodied.

DESCRIPTION OF EXEMPLARY EMBODIMENTS

Hereinafter, some embodiments of the present invention will bedescribed. The embodiments do not limit the invention according to theclaims, and all the combinations of the features described in theembodiments are not necessarily essential to means provided by aspectsof the invention.

FIG. 1 shows a block diagram of an examination apparatus 100 accordingto the present embodiment. The examination apparatus 100 can make iteasy to understand the state of an examination target, by detecting thedifference between an image obtained by capturing the actual state ofthe examination target and an image generated by predicting the statethat the examination target should be in. In the present embodiment, anexample is described in which the examination apparatus 100 uses a probepin for testing a semiconductor as the examination target. However, theexamination target is not limited to this. The examination apparatus 100may be used to analyze an image of bumps of a semiconductor device,examine a pattern of a wiring substrate, or examine other electroniccomponents, or may be used to examine any kind of object that is not anelectrical component.

The examination apparatus 100 may be a computer such as a PC (personalcomputer), tablet computer, smartphone, work station, server computer,or general user computer, or may be a computer system in which aplurality of computers are connected. Such a computer system is also acomputer, in a broad sense. The examination apparatus 100 may beimplemented in a virtual computer environment that can be executed inone or more computers. Instead, the examination apparatus 100 may be aspecialized computer designed for the purpose of examining theexamination target, or may be specialized hardware realized byspecialized circuitry. If the examination apparatus 100 is capable ofconnecting to the Internet, the examination apparatus 100 may berealized by cloud computing.

The examination apparatus 100 includes a target image acquiring section110, a target image masking section 120, a masked region predictingsection 130, a reproduced image generating section 140, a differencedetecting section 150, and a judging section 160.

The target image acquiring section 110 acquires a target image obtainedby capturing an image of the examination target. The target imageacquiring section 110 may acquire an image obtained by pre-processing acaptured image of the examination target, as the target image. In thiscase, the target image acquiring section 110 may acquire the targetimage via a network, acquire the target image via user input, or acquirethe target image via a memory device or the like capable of storingdata, for example. The target image acquiring section 110 supplies theacquired target image to the target image masking section 120 and thedifference detecting section 150.

The target image masking section 120 masks a portion of the targetimage. The target image masking section 120 then supplies the maskedregion predicting section 130 with a plurality of masked images obtainedby masking respectively different portions of the target image.

The masked region predicting section 130 predicts an image of the maskedregion that is masked in the target image. The masked region predictingsection 130 then supplies the reproduced image generating section 140with a plurality of predicted images predicted for each masked region inthe plurality of masked images.

The reproduced image generating section 140 generates a reproduced imageusing the plurality of predicted images predicted respectively for theplurality of masked regions. The reproduced image generating section 140supplies the difference detecting section 150 with the generatedreproduced image.

The difference detecting section 150 detects the difference between thetarget image and the reproduced image. At this time, the differencedetecting section 150 may compare the target image supplied from thetarget image acquiring section 110 to the reproduced image supplied fromthe reproduced image generating section 140, in each predeterminedregion, for example, to calculate a degree of difference in eachpredetermined region. The difference detecting section 150 then suppliesthe calculated degree of difference to the judging section 160.

The judging section 160 judges that the examination target isunacceptable if the degree of difference does not satisfy apredetermined quality standard. The judging section 160 outputs thejudgment result to another function section, another apparatus, and thelike. The following uses a flow to describe the details of examining anexamination target using such an examination apparatus 100.

FIG. 2 shows an example of a flow for examining an examination targetwith the examination apparatus 100 according to the present embodiment.At step 210, the target image acquiring section 110 acquires the targetimage obtained by capturing an image of the examination target. As anexample, the target image acquiring section 110 acquires, via a network,an image of a probe pin for testing a semiconductor capture using anoptical microscope or the like.

The target image acquiring section 110 converts the captured image ofthe examination target to grayscale. The image of the probe pin capturedusing the optical microscope or the like can include three channelscorresponding respectively to the three colors R, G, and B. However,from the viewpoint of the examination according to the presentembodiment, these three channels have approximately the same features,and no one channel has a feature that is unique compared to the otherchannels. Accordingly, the target image acquiring section 110 convertsthe acquired image to a single channel by performing the grayscaleconversion on the acquired image. In this way, the target imageacquiring section 110 may acquire an image obtained by performing agrayscale conversion on the captured image of the examination target, asthe target image. When acquiring the target image, the examinationapparatus 100 can reduce the load of the examination process byperforming the grayscale conversion on the acquired image to only use asingle channel. In a case where it is preferable to use a plurality ofchannels, e.g. a case where the accuracy of the examination is to beimproved, the target image acquiring section 110 may acquire thecaptured image of the examination target as-is as the target image,without performing the grayscale conversion.

The target image acquiring section 110 narrows the captured image of theexamination target. For example, the target image acquiring section 110recognizes the position and size of the probe pin in the image using anobject detection algorithm such as YOLO (You Only Look Once). The targetimage acquiring section 110 narrows the target region by clipping theimage based on the position and size of the recognized probe pin. Inthis way, the target image acquiring section 110 may perform objectdetection of the examination target in the captured image of theexamination target, and acquire the image in which the target region hasbeen narrowed as the target image. In this way, when acquiring thetarget image, the examination apparatus 100 can improve the examinationaccuracy and speed up the examination process by performing objectdetection of the examination target and narrowing the target region. Thetarget image acquiring section 110 acquires the pre-processed image(e.g. the image on which grayscale conversion and narrowing have beenperformed) as the target image, and supplies the target image maskingsection 120 and the difference detecting section 150 with this targetimage.

At step 220, the target image masking section 120 masks a portion of thetarget image. As an example, the target image masking section 120divides the target image acquired at step 210 into a plurality of cells.The target image masking section 120 then sequentially masks one cell ata time among the plurality of cells obtained by dividing the targetimage. The target image masking section 120 supplies the masked regionpredicting section 130 with the plurality of masked images obtained bymasking each of the plurality of cells.

At step 230, the masked region predicting section 130 predicts an imageof a masked region that is masked in the target image. At this time, themasked region predicting section 130 may use a learned model, such as aCNN (Convolution Neural Network) that has learned to be capable ofpredicting an image of the masked region from another image that isunmasked, when the image in which a partial region is masked is input,using only training images of examination targets known to beacceptable, for example. In other words, the masked region predictingsection 130 may use a learned model that has learned to be capable ofpredicting the state of the masked region when the examination target isacceptable, using only training images of examination targets known tobe acceptable. As an example, the masked region predicting section 130inputs each of the plurality of masked images obtained at step 220 intothe learned model, and predicts the image of the masked region for eachof the plurality of cells. The learning of such a model is describedfurther below. The above description is an example in which the maskedregion predicting section 130 uses a learned CNN model, but the presentembodiment is not limited to this. The masked region predicting section130 may predict the image of the masked region using a learned model ofanother algorithm, or may predict the image of the masked region usingan algorithm that is different from learning. The masked regionpredicting section 130 supplies the reproduced image generating section140 with the plurality of predicted images predicted respectively forthe plurality of masked regions.

At step 240, the reproduced image generating section 140 generates areproduced image, using the plurality of predicted images predictedrespectively for the plurality of masked regions. As an example, thereproduced image generating section 140 generates the reproduced imageusing the plurality of predicted images predicted respectively fordifferent cells. At this time, the reproduced image generating section140 may generate the reproduced image by arranging the plurality ofpredicted images predicted respectively for the plurality of cells atstep 230 at the original positions of these cells, for example. Thereproduced image generating section 140 supplies the differencedetecting section 150 with the generated reproduced image.

At step 250, the difference detecting section 150 detects the differencebetween the target image and the reproduced image. As an example, thedifference detecting section 150 makes a comparison between the targetimage supplied from the target image acquiring section 110 and thereproduced image supplied from the reproduced image generating section140, in every predetermined region (e.g. every pixel, every pixel group,every cell used when masking the target image, and the like), andcalculates the degree of difference in every predetermined region. Atthis time, the difference detecting section 150 may calculate the degreeof difference based on the L2 norm, i.e. the Euclidian distance, betweenthe target image and the reproduced image. Furthermore, the differencedetecting section 150 may output a detection map in which the displayattributes (e.g. color, concentration, and the like) in eachpredetermined region differ according to the degree of difference. Thedifference detecting section 150 then supplies the judging section 160with the degree of difference in every predetermined region.

At step 260, the judging section 160 judges whether the degree ofdifference calculated at step 250 is less than or equal to apredetermined threshold value. If the degree of difference is less thanor equal to the predetermined threshold value, i.e. if the degree ofdifference satisfies a predetermined quality standard, the judgingsection 160 proceeds to step 270, judges that the examination target isacceptable, and ends the process. On the other hand, if the degree ofdifference exceeds the predetermined threshold value, i.e. if the degreeof difference does not satisfy the predetermined quality standard, thejudging section 160 proceeds to step 280, judges that the examinationtarget is unacceptable, and ends the process. At this time, the judgingsection 160 may judge the examination target to be unacceptable if thelargest degree of difference, among the degrees of difference for everypredetermined region, exceeds the predetermined threshold value, forexample. The judging section 160 outputs the judgment result to anotherfunction section, another apparatus, and the like. The threshold valueused for this judgment may be the minimum value, or a value slightlylower than this minimum value, obtained when the degree of difference iscalculated by the examination apparatus 100 according to the presentembodiment using an image obtained by capturing an examination targetthat is known to be unacceptable. Furthermore, in the above description,an example is shown in which the judging section 160 judgesacceptability of the examination target based on the largest degree ofdifference among the degrees of differences of every predeterminedregion, but the present embodiment is not limited to this. The judgingsection 160 may judge acceptability of the examination target based onanother statistical value of the degree of difference, e.g. the medianvalue, average value, distribution, or the like.

FIG. 3 shows an example of a target image 310, masked images 320,predicted images 330, and a reproduced image 340 in the examinationusing the examination apparatus 100 according to the present embodiment.The target image acquiring section 110 acquires the target image 310such as shown in this drawing, as an example. The target image maskingsection 120 divides the target image 310 into a plurality of cells 322(a total of 25 cells where [vertical, horizontal]=[1, 1] to [5, 5] inthe present drawing). The target image masking section 120 sequentiallymasks each of these cells 322 using a mask 324, to generate each of theplurality of masked images 320. As an example, the top, middle, andbottom masked images 320 in the present drawing are respectively caseswhere the cell [2, 3], the cell [3, 3], and the cell [4, 3] are masked.The masked region predicting section 130 predicts the image of themasked region 332 for each of the plurality of cells 322, to generateeach of the plurality of predicted images 330. The reproduced imagegenerating section 140 generates the reproduced image 340 by arrangingthe plurality of predicted images 330 predicted for the plurality ofcells 322 at the original positions of these cells 322.

FIG. 4 shows an example of the examination result in a case where theexamination target is acceptable, in the present embodiment. If theexamination target is acceptable, the reproduced image 340 generated bythe examination apparatus 100 is approximately the same as the targetimage 310. This is because, since the examination apparatus 100 predictsthe state that the masked region should be in if the examination targetis acceptable using the learned model that has learned using onlytraining images of examination targets that are known to be acceptable,the reproduced image 340 generated by the examination apparatus 100 isan image in which is reproduced the state that the examination targetshould be in if this examination target is acceptable. Accordingly, evenin the detection map 400 in which the display attributes of everypredetermined region differ according to the degree of difference, allof the regions have approximately the same display attributes.Furthermore, in a distribution (bottom portion of the present drawing)obtained by counting the number of unit regions (cell unit regions inthe present drawing) of every degree of difference, the majority ofcells are counted as having a degree of difference near 0, and none ofthe cells are counted at positions where the degree of difference isgreater than 1.5 (the threshold value). Here, the unit regions may beregions such as pixel units, pixel group units, and cell units, and thepresent drawing shows an example in which the unit regions are cell unitregions. Furthermore, a greater width on the vertical axis indicates agreater number of counted cells. The examination apparatus 100 judgesthe examination target in the target image 310 to be acceptable if thedifference between the target image 310 and the reproduced image 340 issmall in this manner.

FIG. 5 shows an example of examination results in a case where theexamination target is unacceptable. If the examination target isunacceptable (e.g. if the examination target is cracked), the reproducedimage 340 generated by the examination apparatus 100 is different fromthe target image 310. In this case, regions where the degree ofdifference differs are shown by differing attributes in the detectionmap 400. Furthermore, in the distribution of the degrees of difference,many cells are counted at positions where the degree of difference islarge, and several cells are counted at positions where the degree ofdifference is greater than 1.5. When the difference between the targetimage 310 and the reproduced image 340 is large in this manner, theexamination apparatus 100 judges the examination target in the targetimage 310 to be unacceptable.

In this way, according to the examination apparatus 100 of the presentembodiment, it is possible to easily understand the state of theexamination target by detecting the difference between an image (targetimage 310) obtained by capturing the actual state of the examinationtarget and an image (reproduced image 340) obtained by predicting andreproducing the state that the examination target should be in if theexamination target is acceptable. Furthermore, the examination apparatus100 outputs the detection map 400 in which the display attributes differin every region according to the degree of difference, and therefore itis possible to easily understand the location of defects in theexamination target. Yet further, the examination apparatus 100 outputsthe distribution obtained by counting the number of unit regions ofevery degree of difference, and therefore it is possible to understandthe frequency with which regions having different degrees of differenceoccur in the image.

Such an examination using the examination apparatus 100 according to thepresent embodiment may be performed during manufacturing of the probeand before shipping of the probe, or may be performed immediately beforeor at intervals of the actual testing of a semiconductor device usingthe probe, for example. Furthermore, in a case where a visual inspectionusing images from a plurality of directions would be useful, theexamination apparatus 100 may more deeply understand the state of theexamination target by performing this examination using the images ofthe examination target captured from a plurality of directions.

The description above uses an example in which the examination apparatus100 judges the acceptability of the examination target based only on anappearance examination using an image, but the present embodiment is notlimited to this. The examination apparatus 100 may judge theacceptability of the examination target based on both an appearanceexamination using an image and the electrical characteristics.

As an example, if the judging section 160 judges that the examinationtarget is unacceptable, the electrical characteristics of theexamination target are predicted from the target image obtained bycapturing an examination target that has been judged to be unacceptable,and if these electrical characteristics do not satisfy a predeterminedquality standard, the examination target may be confirmed as beingunacceptable. For example, the judging section 160 may use a learnedmodel that has learned to be able to predict the electricalcharacteristics of the examination target when the target image obtainedby capturing the examination target is input thereto. In other words,the judging section 160 may use a learned model that has learned to becapable of predicting the electrical characteristics, e.g. theresistance value and the like, of a probe pin when the target imageobtained by capturing a probe pin is input thereto. Then, if the probepin is judged to be unacceptable at step S280 of FIG. 2, the judgingsection 160 may input the target image obtained by capturing a probe pinjudged to be unacceptable to this learned model to predict theresistance value of the probe pin, and if the resistance value does notsatisfy a predetermined quality standard, may confirm that the probe pinthat is the examination target is unacceptable. The description aboveshows an example in which the judging section 160 predicts theelectrical characteristics of the examination target using a learnedmodel, but the present embodiment is not limited to this. The judgingsection 160 may predict the electrical characteristics of theexamination target from the target image using an algorithm differentfrom learning.

Similarly, if the examination target is judged to be unacceptable, thejudging section 160 may predict the electrical characteristics of theexamination target from the target image obtained by capturing theexamination target that has been judged to be unacceptable, and if theelectrical characteristics fulfill the predetermined quality standard,may determine that the examination target is acceptable. For example, ifthe probe pin is judged to be unacceptable at step S280 of FIG. 2, thejudging section 160 may input the target image obtained by capturing theprobe pin judged to be unacceptable to the learned model to predict theresistance value of the probe pin, and if the resistance value satisfiesa predetermined quality standard, may determine that the probe pin thatis the examination target is acceptable.

In this way, the examination apparatus 100 can accurately judge theacceptability of the examination target by considering the electricalcharacteristics, in addition to the appearance examination using animage. For example, even if an examination target that has been judgedto be unacceptable in the appearance examination using an image due tothe effects of reflected shadows, particles, or the like during theimage capturing of the examination target, there can be cases wherethere are no problems with the electrical characteristics. Therefore,the examination apparatus 100 can more accurately judge theacceptability of the examination target by combining this appearanceexamination with the electrical characteristics.

FIG. 6 shows an example of a block diagram of a learning apparatus 600according to the present embodiment. The learning apparatus 600 causes amodel to learn to be capable of predicting a masked region from an imageof another region that is not masked, when an image in which a partialregion is masked is input thereto, using only training images of targetsthat are known to be acceptable. In other words, the learning apparatus600 causes a model to be able to predict the state that a partial regionthat is masked should be in if the examination target is acceptable,using only training images of targets known to be acceptable. Theexamination apparatus 100 according to the present embodiment maypredict the masked region using a learned model that is caused to learnby the learning apparatus 600 such as shown in the present drawing, forexample.

The learning apparatus 600 may be a computer such as a PC (personalcomputer), tablet computer, smartphone, work station, server computer,or general purpose computer, or may be a computer system in which aplurality of computers are connected. Such a computer system is also acomputer, in a broad sense. The learning apparatus 600 may beimplemented in a virtual computer environment that can be executed inone or more computers. Instead, the learning apparatus 600 may be aspecialized computer designed for the purpose of model learning, or maybe specialized hardware realized by specialized circuitry. If thelearning apparatus 600 is capable of connecting to the Internet, thelearning apparatus 600 may be realized by cloud computing.

The learning apparatus 600 includes a training image acquiring section610, a training image masking section 620, a predictive model 630, anerror calculating section 640, and a model updating section 650.

The training image acquiring section 610 acquires a training image. Asan example, the training image acquiring section 610 may acquire aplurality of images in which examination targets known to be acceptableare captured, as training images. At this time, the training imageacquiring section 610 may acquire the training images via a network,acquire the training images via user input, or acquire the trainingimages via a memory device or the like capable of storing data, forexample. The training image acquiring section 610 supplies the trainingimage masking section 620 and the error calculating section 640 with theacquired training images.

The training image masking section 620 masks a portion of a trainingimage. As an example, the training image masking section 620 suppliesthe predictive model 630 with a plurality of masked images obtained byrandomly masking the plurality of images acquired as training images.

The predictive model 630 receives a training image in which a partialregion is masked, and outputs a model image obtained by predicting thetraining image. As an example, when the training image in which thepartial region is masked is input, the predictive model 630 predicts thestate that the masked region should be in if the examination target isacceptable, and outputs the model image obtained by predicting thistraining image. At this time, the predictive model 630 may use analgorithm such as CNN, for example, to predict the masked region. Theabove description shows an example in which the predictive model 630uses CNN, but the present embodiment is not limited to this. Thepredictive model 630 may predict the image of the masked region using analgorithm other than CNN. The predictive model 630 supplies the errorcalculating section with the output model image.

The error calculating section 640 calculates the error between thetraining image supplied from the training image acquiring section 610and the model image supplied from the predictive model 630. The errorcalculating section 640 supplies the model updating section 650 with thecalculated error.

The model updating section 650 updates the predictive model 630 based onthe error between the training image supplied from the training imageacquiring section 610 and the model image supplied from the predictivemodel 630. The following uses a flow to describe the details of thelearning of a model using such a learning apparatus 600.

FIG. 7 shows an example of a flow of the learning of the predictivemodel 630 by the learning apparatus 600 according to the presentembodiment. At step 710, the training image acquiring section 610acquires the training images. As an example, the training imageacquiring section 610 acquires, via a network, a plurality of images inwhich examination targets known to be acceptable are captured. Thetraining image acquiring section 610 acquires the images obtained bypre-processing the acquired images, as the training images, in the samemanner as the target image acquiring section 110. At this time, thetraining image acquiring section 610 may drop images in which the focusis shifted from the examination target, using a contour detection filteror the like, without acquiring these images as training images. Thetraining image acquiring section 610 supplies the training image maskingsection 620 and the error calculating section 640 with the acquiredtraining images.

At step 720, the training image masking section 620 masks portions ofthe training images. For example, the training image masking section 620randomly selects a plurality of images acquired as training images.Then, the training image masking section 620 randomly masks one cell,among a plurality of cells obtained by dividing the image region, ineach of the randomly selected images. The training image masking section620 then supplies the predictive model 630 with the plurality of maskedimages obtained by randomly masking the randomly selected images.

At step 730, the predictive model 630 receives the masked trainingimages, and outputs a model image obtained by predicting the trainingimages. For example, when the masked images in which partial regions arerandomly masked are input, the predictive model 630 predicts the imagesof the masked regions from the images of other regions that are notmasked. The predictive model 630 then outputs the model image byembedding the predicted images in the masked regions of the trainingimages. The predictive model 630 supplies the error calculating section640 with the model image.

At step 740, the error calculating section 640 calculates the errorbetween the training image supplied from the training image acquiringsection 610 and the model image supplied from the predictive model 630.The error calculating section 640 supplies the model updating section650 with the calculated error.

At step 750, the model updating section 650 updates the predictive model630 based on the error between the training image supplied from thetraining image acquiring section 610 and the model image supplied fromthe predictive model 630. For example, the model updating section 650updates parameters such as weights in the predictive model 630 in amanner to minimize an objective function, which is the error calculatedat step 740.

At step 760, the learning apparatus 600 judges whether the training hasended. At step 760, if it is judged that the training has not ended, thelearning apparatus 600 returns the processing to step 710 and repeatsthese processes. On the other hand, if it is judged at step 760 that thetraining has ended, the learning apparatus 600 ends the processing. Atthis time, the learning apparatus 600 may judge whether the training hasended based on conditions such as the training time, number oftrainings, and training accuracy, for example.

In this way, according to the learning apparatus 600 of the presentembodiment, the predictive model 630 is updated in a manner to minimizethe error between the training image and the model image, using only aplurality of images in which the captured examination targets are knownto be acceptable as the training images, and therefore the learningapparatus 600 can update the predictive model 630 to be capable topredicting the image of a state that the masked region should be in ifthe examination target is acceptable.

Various embodiments of the present invention may be described withreference to flowcharts and block diagrams whose blocks may represent(1) steps of processes in which operations are performed or (2) sectionsof apparatuses responsible for performing operations. Certain steps andsections may be implemented by dedicated circuitry, programmablecircuitry supplied with computer-readable instructions stored oncomputer-readable media, and/or processors supplied withcomputer-readable instructions stored on computer-readable media.Dedicated circuitry may include digital and/or analog hardware circuitsand may include integrated circuits (IC) and/or discrete circuits.Programmable circuitry may include reconfigurable hardware circuitscomprising logical AND, OR, XOR, NAND, NOR, and other logicaloperations, flip-flops, registers, memory elements, etc., such asfield-programmable gate arrays (FPGA), programmable logic arrays (PLA),and the like.

The computer-readable medium may be a tangible device that can storeinstructions to be executed by a suitable device, and as a result, acomputer-readable medium having instructions stored thereon is a productthat includes instructions that can be executed in order to create themeans for executing the operations designated by flow charts and blockdiagrams. Examples of the computer-readable medium may include anelectronic storage device, a magnetic storage device, an optical storagedevice, an electromagnetic recording medium, a magnetic recordingmedium, an optical recording medium, an electromagnetic recordingmedium, a semiconductor recording medium, and the like. Specificexamples of the computer-readable medium may include a floppy(Registered Trademark) disk, a diskette, a hard disk, a random accessmemory (RAM), a read-only memory (ROM), an erasable programmableread-only memory (EPROM or Flash memory), an electrically erasableprogrammable read-only memory (EEPROM), a static random access memory(SRAM), a portable compact disc read-only memory (CD-ROM), a digitalversatile disk (DVD), a Blu-ray (Registered Trademark) disk, a memorystick, an integrated circuit card, or the like.

The computer-readable instructions may be assembler instructions,instruction-set-architecture (ISA) instructions, machine instructions,machine dependent instructions, microcode, firmware instructions,state-setting data, or either source code or object code written in anycombination of one or more programming languages, including an objectoriented programming language such as Smalltalk, JAVA (RegisteredTrademark), Javascript (Registered Trademark), C++ or the like, andconventional procedural programming languages, such as the “C”programming language or similar programming languages.

The computer-readable instructions may be provided to a processor orprogrammable circuitry of a general purpose computer, special purposecomputer, or other programmable data processing apparatus to produce amachine, either locally, via a local area network (LAN), or via a widearea network (WAN) such as the Internet, and may be executed to createthe means for performing the operations designated by the flow chartsand block diagrams. Examples of the processor include a computerprocessor, a processing unit, a microprocessor, a digital signalprocessor, a controller, a microcontroller, and the like.

FIG. 8 shows an example of a computer 2200 in which aspects of thepresent invention may be wholly or partly embodied. A program that isinstalled in the computer 2200 can cause the computer 2200 to functionas or perform operations associated with apparatuses of the embodimentsof the present invention or one or more sections thereof, and/or causethe computer 2200 to perform processes of the embodiments of the presentinvention or steps thereof. Such a program may be executed by the CPU2212 to cause the computer 2200 to perform certain operations associatedwith some or all of the blocks of flowcharts and block diagramsdescribed herein.

The computer 2200 according to the present embodiment includes a CPU2212, a RAM 2214, a graphic controller 2216, and a display device 2218,which are mutually connected by a host controller 2210. The computer2200 also includes input/output units such as a communication interface2222, a hard disk drive 2224, a DVD-ROM drive 2226 and an IC card drive,which are connected to the host controller 2210 via an input/outputcontroller 2220. The computer also includes legacy input/output unitssuch as a ROM 2230 and a keyboard 2242, which are connected to theinput/output controller 2220 through an input/output chip 2240.

The CPU 2212 operates according to programs stored in the ROM 2230 andthe RAM 2214, thereby controlling each unit. The graphic controller 2216obtains image data generated by the CPU 2212 on a frame buffer or thelike provided in the RAM 2214 or in itself, and causes the image data tobe displayed on the display device 2218.

The communication interface 2222 communicates with other electronicdevices via a network. The hard disk drive 2224 stores programs and dataused by the CPU 2212 within the computer 2200. The DVD-ROM drive 2226reads the programs or the data from the DVD-ROM 2201, and provides thehard disk drive 2224 with the programs or the data via the RAM 2214. TheIC card drive reads programs and data from an IC card, and/or writesprograms and data into the IC card.

The ROM 2230 stores therein a boot program or the like executed by thecomputer 2200 at the time of activation, and/or a program depending onthe hardware of the computer 2200. The input/output chip 2240 may alsoconnect various input/output units via a parallel port, a serial port, akeyboard port, a mouse port, and the like to the input/output controller2220.

A program is provided by computer readable media such as the DVD-ROM2201 or the IC card. The program is read from the computer readablemedia, installed into the hard disk drive 2224, RAM 2214, or ROM 2230,which are also examples of computer readable media, and executed by theCPU 2212. The information processing described in these programs is readinto the computer 2200, resulting in cooperation between a program andthe above-mentioned various types of hardware resources. An apparatus ormethod may be constituted by realizing the operation or processing ofinformation in accordance with the usage of the computer 2200.

For example, when communication is performed between the computer 2200and an external device, the CPU 2212 may execute a communication programloaded onto the RAM 2214 to instruct communication processing to thecommunication interface 2222, based on the processing described in thecommunication program. The communication interface 2222, under controlof the CPU 2212, reads transmission data stored on a transmissionbuffering region provided in a recording medium such as the RAM 2214,the hard disk drive 2224, the DVD-ROM 2201, or the IC card, andtransmits the read transmission data to a network or writes receptiondata received from a network to a reception buffering region or the likeprovided on the recording medium.

In addition, the CPU 2212 may cause all or a necessary portion of a fileor a database to be read into the RAM 2214, the file or the databasehaving been stored in an external recording medium such as the hard diskdrive 2224, the DVD-ROM drive 2226 (DVD-ROM 2201), the IC card, etc.,and perform various types of processing on the data on the RAM 2214. TheCPU 2212 may then write back the processed data to the externalrecording medium.

Various types of information, such as various types of programs, data,tables, and databases, may be stored in the recording medium to undergoinformation processing. The CPU 2212 may perform various types ofprocessing on the data read from the RAM 2214, which includes varioustypes of operations, processing of information, condition judging,conditional branch, unconditional branch, search/replace of information,etc., as described throughout this disclosure and designated by aninstruction sequence of programs, and writes the result back to the RAM2214. In addition, the CPU 2212 may search for information in a file, adatabase, etc., in the recording medium. For example, when a pluralityof entries, each having an attribute value of a first attributeassociated with an attribute value of a second attribute, are stored inthe recording medium, the CPU 2212 may search for an entry matching thecondition whose attribute value of the first attribute is designated,from among the plurality of entries, and read the attribute value of thesecond attribute stored in the entry, thereby obtaining the attributevalue of the second attribute associated with the first attributesatisfying the predetermined condition.

The above-explained program or software modules may be stored in thecomputer readable media on or near the computer 2200. In addition, arecording medium such as a hard disk or a RAM provided in a serversystem connected to a dedicated communication network or the Internetcan be used as the computer readable media, thereby providing theprogram to the computer 2200 via the network.

While the embodiments of the present invention have been described, thetechnical scope of the invention is not limited to the above describedembodiments. It will be apparent to persons skilled in the art thatvarious alterations and improvements can be added to the above-describedembodiments. It should also apparent from the scope of the claims thatthe embodiments added with such alterations or improvements are withinthe technical scope of the invention.

The operations, procedures, steps, and stages of each process performedby an apparatus, system, program, and method shown in the claims,embodiments, or diagrams can be performed in any order as long as theorder is not indicated by “prior to,” “before,” or the like and as longas the output from a previous process is not used in a later process.Even if the process flow is described using phrases such as “first” or“next” in the claims, embodiments, or diagrams, it does not necessarilymean that the process must be performed in this order.

What is claimed is:
 1. An examination apparatus comprising: a target image acquiring section that acquires a target image obtained by capturing an examination target image; a target image masking section that masks a portion of the target image, the target image masking section being configured to divide the target image into a plurality of cells, each cell corresponding to a predetermined region that is to be masked, the target image masking section being configured to generate a plurality of masked images by sequentially masking each of the plurality of cells to generate a corresponding one of the plurality of masked images, such that each one of the plurality of masked images corresponds to a respective one of the plurality of cells and each one of the plurality of masked images has a predetermined masked region defined by the respective one of the plurality of cells; a masked region predicting section that predicts a predicted image of the masked region in each one of the plurality of masked images to generate a plurality of predicted images predicted respectively for the masked region in each of the plurality of masked images; a reproduced image generating section that generates a reproduced image using the plurality of predicted images by placing each of the plurality of predicted images in the same position in the reproduced image as the position of a corresponding one of the plurality of cells in the target image; and a difference detecting section that detects a difference between the target image and the reproduced image.
 2. The examination apparatus according to claim 1, wherein the difference detecting section compares the target image to the reproduced image in every predetermined region, to calculate a degree of the difference in every predetermined region.
 3. The examination apparatus according to claim 2, further comprising: a judging section that judges the examination target to be unacceptable if the degree of difference does not satisfy a predetermined quality standard.
 4. The examination apparatus according to claim 3, wherein the judging section judges the examination target to be unacceptable if a largest degree of difference, among the degrees of difference of every predetermined region, exceeds a predetermined threshold value.
 5. The examination apparatus according to claim 3, wherein if the judging section judges the examination target to be unacceptable, the judging section predicts an electrical characteristic of the examination target from the target image obtained by capturing the examination target that was judged to be unacceptable, and confirms that the examination target is unacceptable if the electrical characteristic does not satisfy a predetermined quality standard.
 6. The examination apparatus according to claim 3, wherein if the judging section judges the examination target to be unacceptable, the judging section predicts an electrical characteristic of the examination target from the target image obtained by capturing the examination target that was judged to be unacceptable, and determines that the examination target is acceptable if the electrical characteristic satisfies a predetermined quality standard.
 7. The examination apparatus according to claim 2, wherein the difference detecting section outputs a detection map in which a display attribute differs in every predetermined region, according to the degree of difference.
 8. The examination apparatus according to claim 2, wherein the difference detecting section calculates the degree of difference based on a Euclidian distance between the target image and the reproduced image.
 9. The examination apparatus according to claim 1, wherein the target image acquiring section acquires an image obtained by performing a grayscale conversion on the captured image of the examination target, as the target image.
 10. The examination apparatus according to claim 1, wherein the target image acquiring section acquires an image obtained by performing object detection on the examination target in the captured image of the examination target to narrow a target region, as the target image.
 11. An examination method comprising: acquiring a target image obtained by capturing an examination target image; sequentially masking a plurality of predetermined regions of the target image to generate a plurality of masked images each having a masked region corresponding to a respective one of the plurality of predetermined regions; predicting a predicted image of the masked region of each of the plurality of masked images to generated a plurality of predicted images; generating a reproduced image using the plurality of predicted images by placing each of the plurality of predicted images in the same position in the reproduced image as the position of the respective one of the plurality of predetermined regions in the target image; and detecting a difference between the target image and the reproduced image.
 12. A non-transitory computer-readable medium storing thereon an examination program that, when executed by a computer, causes the computer to function as: a target image acquiring section that acquires a target image obtained by capturing an examination target image; a target image masking section that masks a portion of the target image, the target image masking section being configured to divide the target image into a plurality of cells, each cell corresponding to a predetermined region that is to be masked, the target image masking section being configured to generate a plurality of masked images by sequentially masking each of the plurality of cells to generate a corresponding one of the plurality of masked images, such that each one of the plurality of masked images corresponds to a respective one of the plurality of cells and each one of the plurality of masked images has a predetermined masked region defined by the respective one of the plurality of cells; a masked region predicting section that predicts a predicted image of the masked region in each one of the plurality of masked images to generate a plurality of predicted images predicted respectively for the masked region in each of the plurality of masked images; a reproduced image generating section that generates a reproduced image using the plurality of predicted images by placing each of the plurality of predicted images in the same position in the reproduced image as the position of a corresponding one of the plurality of cells in the target image; and a difference detecting section that detects a difference between the target image and the reproduced image.
 13. The examination apparatus according to claim 1, wherein the plurality of cells are of uniform size and shape.
 14. The examination method according to claim 11, wherein the plurality of predetermined regions are of uniform size and shape.
 15. The non-transitory computer-readable medium according to claim 12, wherein the plurality of cells are of uniform size and shape. 